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; I ■ Abstract: We study the estimation of two- type continuous-state branching processes with immi- 

Qh! gration (CBI-processes). The ergodicity of the processes is proved. We also establish the strong 

■ consistency and central limit theorems of the conditional least squares estimators and the weighted 
conditional least squares estimators of the drift and diffusion coefficients based on low frequency 
observations. 

Key words and phrases: Two-type, continuous-state branching process with immigration, stochas- 

■ tic differential equation, conditional least squares estimator, weighted conditional least squares 
P I | estimator, consistency, central limit theorem 

S3 ■ 1 Introduction 



Branching processes have been used widely not only in biology, but also in financial world. For example, 
, Galton- Watson branching processes with immigration (GWI-processes) are used to study the evolution 

of different species. Continuous-state branching processes (CB-proccsses) were first introduced by Jifina 
(1958). In particular, a continuous CB-process can be obtained as the unique solution of a stochastic 
equation system driven by Brownian motion. Kawazu and Watanabc (1971) constructed continuous-state 
\jf~} , branching processes with immigration (CBI-processes). In view of the results of Dawson and Li (2006), a 

' general single-type CBI-process is the unique strong solution of a stochastic equation driven by Brownian 

motions and Poisson random measures. The two-type CB-processes was first be introduced by Watanabe 
CO ' (1969). Ma (2012) proved the existence and uniqueness of the strong solution of a two-dimensional 

stochastic integral equation system with jumps. He also showed that the unique solution is a two-type 
CBI-process. In financial world, multitype CBI-processes are used to describe the relations of the prices 
of different assets and interest rates of different currencies. 

X . . . 

■ Firstly, we introduce a special continuous single-type CBI-process defined by the following equation: 

dX t = (a- bX t )dt + ay/X~ t dB t , (1.1) 

where (a,b,a) € (0, +oo) 3 and B t is a standard Brownian motion. In fact, the solution of (jl.ip is 
also known as the Cox-Ingersoll-Ross model (CIR-model) introduced by Cox et al. (1985) for the term 
structure of interest rates. The above equation was also studied in Ikeda and Watanabe (1989) and Revuz 
and Yor (1991). The basic theory of general CBI-processes was developed in Li (2011). The appealing 
properties of this process are as follows: 

(1) The process stays nonnegativc. 

(2) It converges to a steady-state law with mean a/b, the so-called long-term value, with speed of 
adjustment b. 

(3) The incremental variance is proportional to its current value. 

However, the one-dimensional CIR-model doesn't describe the connection among interest rates of different 
currencies. In order to give more objective description to the financial environment, we need to deal with 
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multifactor CIR-model or the multitype CBI-processcs. In order to make the presentation easier, we just 
analyze the two-type CBI-processcs defined by the following equation: 



f dXiit) = (ax - 611X1 (t) + b 12 X 2 (t))dt + a 1 ^ /X^) dB 1 {t), 
\ dX 2 {t) = (a 2 + b 21 X 1 (t)-b 22 X 2 (t))dt + a 2 ^/X 2 Jt)dB 2 (t), 

where 9 — (ai, a 2 , 611, 612, 621, 622, € (0, +00) 3 x [0, oo) 2 x (0, oo) 3 , and Bi(t), i = 1,2 are indepen- 

dent standard Brownian motions. The existence and uniqueness of the solution to (|1.2j) were proved in 
Ma (2012). We can rewrite (| 1 . 2 j) into the vector form: 



where 




dX t = (A - BX t )dt + Z^/XtdWt, (1.3) 



B = 





We can use a special form of (|1.2[) to describe the relations among the interest rates of different 
currencies. In currency market, we assume Xi(t) is the interest rate of a very strong and influential 
currency, and X 2 (t) is the interest rate of a less influential currency. The situation can be described by 
the following stochastic equation: 



dX^t) = M£r -Xiitydt + ^y/xJtjdBxit), 



\ dX 2 (t) = b 22 (f^ + b ^X 1 {t)-X 2 (t))dt + a 2 ^X 2 {£)dB 2 {t). 

Here, the first equation gives the evolution of X±(t), which is just a one-dimensional CIR- model. The 
second equation describes the evolution of X 2 (t), which is affected not only by the random noise, but 
also by X\(t). Specifically, the second coordination X 2 (t) has the following properties: 

(1) It stays nonnegative. 

(2) It converges to a steady-state law with mean a 2 /b 22 + (621/622) (osi/611) (this can be easily got from 
(|3.1j) with t — > 00), the so-called long-term value, where the second term is contributed by Xi(t). 



(3) Its incremental variance is proportional to its current value. 

If 612 and 621 are neither zero, then (jl.2p can account for the fluctuations of the rate of two currencies 
that affect with each other. Furthermore, (|1.2[) can also be used to analyze the corporate profitability 
and the market yield. For example, 

(1) In a perfectly competitive market, we can use (|1.2j) with 612 = to analyze the relationship between 
the corporate profitability and the yield of its corresponding industry. 

(2) In an oligopoly market, we can use (|1.2|) to describe the relationship among profitability of the 
different enterprises. 

(3) However, in pure monopoly market, the one-dimensional CIR-model is suitable enough to analyze 
the corporate profitability or the market yield. 

However, before using (|1.3j) to solve the practical problems, we need to estimate the parameters in 
the equation based on the historical information. For the single-type CBI-processes, the approaches to 
parameter estimation can be found in Long-staff and Schwartz (1992) and Bibby and S0rensen (1995). 
Overbeck and Ryden (1997) also gave the conditional least squares estimators (CLSEs). Estimators of 
the matrix of offspring means and the vector of stationary means in a multitype GWI-process had been 
given in Quine and Durham (1977). For multitype GWI-process, the weighted conditional least squares 
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estimator (WCLSE) of the mean matrix was developed in Shete and Sriram (2003). The asymptotically 
properties of CLSEs of GWI-processes with general offspring laws were studied in Venkataraman (1982) 
and Wei and Winnicki (1989). The asymptotics of CLSEs and WCLSEs of a stable CIR-model was studied 
in Li and Ma (2013). It is well-known that the CBI-processes are special examples of the affine Markov 
processes studied in Duffie et al. (2003). The ergodicity and estimation of some different two-dimensional 
affine processes were studied in Barczy et al. (2013a, 2013b, 2013c). 

In this work, we give the CLSEs and the WCLSEs of the parameters in (|1.3p using low frequency 
observations at equidistant time points {kA : k = 0, 1, . . . , n} of a single realization {X t : t > 0}, where 
X t = (Xi(t), X2(t)) T . For simplicity, we take A = 1, but all the results presented below can be modified 
to the general case. This is based on the minimization of a sum of squared deviation about conditional 
expectations developed in Klimko and Nelson (1978), who applied their results to the CLSEs of the 
offspring and immigration means of subcritical GWI-processes. Then, as Overbeck and Ryden (1997), 
we shall study the consistency and the central limit theorems of CLSEs and WCLSEs. 

The paper is organized as follows. In Section 2, we give the ergodicity of the two-type CBI-process, 
which is essentially necessary for the study of the estimators. Section 3 is devoted to the study of the 
CLSEs and WCLSEs of (A, B) and E. The consistency and asymptotic normality of the CLSEs and 
WCLSEs are given in the Section 4. All the proofs are given in Section 5. 



2 Multitype CBI-processes and ergodicity 

In this section, firstly we give the definition and a few properties of two-type CBI-processes. In particular, 
we show that the solution of (|1.3[) is a two-type CBI-process. Secondly, we show that the two-type CBI- 
process is crgodic under a weak condition. These results are very important to study the consistency and 
asymptotic normality of the estimators. 

Let D = [0,oo) 2 . In view of (|1.3jl . we consider the branching mechanisms <pi, i = 1,2, with represen- 
tation: 



(2.1) 



01 (A) = 6nA 1 -6 12 A 2 + ^A?, 

°2 \2 

where A = (Ai, A2) T G D. Next we give the definition of the two- type CBI-processes. 



Definition 2.1 A Markov process X t = (X\ (t), Xi(t)) in D is called a two-type continuous-state branch- 
ing process with immigration (CBI-process), if it has transition semigroup (Qt)t>o given by 



e-<V' x >Q t (x,dy) = exp l-(x,v t {\)) -J (A,v s (X))dsj , A,z e D, (2.2) 
where A is given in SI. 3]) and v t (X) = (vi(t, X),V2(t, A)) T is the unique solution of 



&«i(t,A) = -Mvt{X)) = -b 11 v 1 (t,X) + b 1 2V 2 (t,X)-^v 1 (t,X) 2 , 



|«a(t,A) = -Mvt{X))=b2iv 1 {t,X)-b 2 2V2(t,X)-^v 2 (t,X) 2 



(2.3) 



with the initial condition Vq(X) = X £ D. 



By Theorem 2.3 in Ma (2012), there is a unique non-negative weak solution to (|1.3[) and the solution is 
a CBI-process with transition semigroup {Qt)t>o defined by (|2.2[) . He also showed that there is a unique 
non- negative strong solution to (|1.3|l . 

With the conclusions above, The following theorem gives a necessary and sufficient condition for the 
ergodicity of the semigroup (Qt)t>o- 

Let k = gj^b^ ■ Then we get the following conclusion, the proof will be given in Section 5. 
Theorem 2.2 Suppose both of the eigenvalues of B = ( 11 12 ) are positive, i.e. K < 1. Then 



~b2i 



>22 
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the transition semigroup (Qt)t>o has the unique stationary distribution Qoo given by 

e-^Q^idy) = exp j- J (A, v s (X))ds\ . (2.4) 
Moreover, for every x € D, we have Qt(x, ■) — > Qoo by weak convergence as t — > oo. 

With Theorem 2.2 the following corollary can be easily proved like the proof of Theorem 2.7 in Li 
and Ma (2013). 

Corollary 2.3 Suppose the conditions of Theorem 2.2 are satisfied, then X t is mixing and it's tail o~- 
algebra is trivial, e.g., Durrett (2010). 

The results of this paper on the asymptotics of the estimators will be derived under the assumption 
that the eigenvalues of B arc all positive and X$ is distributed according to the stationary law. By 
Birkhoff's ergodic theorem, X t is a stationary and ergodic process, but by a fairly simple (continuous 
time) coupling argument it can be seen that they are valid for arbitrary initial distributions. 



3 CLSEs and WCLSEs of (A, B) and £ 



Recall A = 1. Let {,^ k = a({X , . . . , X k }) : k = 0, 1,2, ••■}. In this section we give the CLSEs 
and WCLSEs of the drift coefficients (A, B) and the diffusion coefficients S based on the observations 
{X k :k = 0,...,n}. 

Following Klimko and Nelson (1978) and Overbeck and Ryden (1997), we first define the CLSE of 
(A,B). The basic ideas are explained as follows. By applying Ito's formula to (|1.3p . for any t > r > 
we have 



X t = e-^-^Xr + f e- B(t -^Ads+ f e- B ^T,^/T s dW s . 

J r J r 

Apparently, J e~ B ^~ s ^Tly/X^dW s is a local martingale with respect to {^t}- For 



(3.1) 



E 









o%X 2 {t) 



-B L (t-s] 



dt 



< oo, V t > 0, 



then f e B< - t s ^Y.y/XldW s is a martingale. 
Let I be the identical matrix and define 

p = B-\l-e~ B )A, 7 = e~ B . (3.2) 

From (|3.1jl and (|3.2j) we can easily obtain the stochastic regressive equation 

Xk = p + 7-Xfc-i + £k, (3-3) 

where 

e k = / e- B{k - s ^^/T s dW s . 
Jk-i 

Apparently, {e k : fc > 0} is a martingale differential sequence with respect to {,^ k '■ k > 0}. Let 
m{x; A, B) = p + 72, then 

Ee[X k \X k -i} = m(Xk-i;A,B). 
Suppose g(x) is a Borel measurable function on M. 2 , satisfying that 

V(\g(X Q )\ 2 ) < 00, 
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let g k = g(Xk-i), where k = 1, . . . ,n. The WCLSE of (A, B) can be given by minimizing the sum of 
squares 

n n 

9keje k = 9k(X k -p- 1 X k _ 1 j T {X k -p- 1 X k ^ 1 ). (3.4) 

k=l k=l 

In particular, the estimators of (p, 7) are given by 

Pn = 9n (x n - %X n ^j (3.5) 



and 



where 



In = (Tl, n - T ltn ) ■ (T 2 , n ~ ?2,n) \ (3.6) 
_^ n 1 n 1 71 

X n = -\g k X k , X n = -\g k X k ^ 1 , g n = - } g k , 

k=l k=l k=l 

1 ™ 

Tin = — > (gkX k — X n )(g k X k -i — X n ) , 
n £ — ' 

k=l 

_^ n 1 n 
Tl, n = — / ](9k — 9n){9kXkX k T _ 1 \j5feXfeXj_j), 

k=l k=l 

1 ™ 

Tin = / {gkX k -\ — X n )(g k X k _i — X n ) , 
n * — ' 

k=l 

^ n 1 n 

T 2 ,n = - Y](.9fe - gn)(gkX k -iX k r _ 1 V.9/.A-lX t T _ 1 ). 

k=l k=l 

According to (|3.2p . we can easily get the CLSE of (A, B) by simple calculation, 

An = (I - %)~ x B n p n , 
B n = -\a%, 

oo 

Where ln7„ = (~ ^-) k ~ 1 \iln — l) fc - From the proof of Theorem 4.1, j n 7 and the spectral radius 

fe=0 

of 7 is less than one, so B n is reasonable. 

We now turn to the estimation of S. For the connection between S and (ui,^), we just need to 
study (<7i,(72). For this, we need to compute 

w(* fc _i;0) =E 9 [(X fe -m(X fc _ 1 ;A, J B))(X fc -m(X fe _ 1 ;A,B)) T |X fe _ 1 ]. (3.7) 

By applying Ito's formula to X t Xj , for any t > r > we have 

X t Xj = X r Xj + J [X S A T + AXj + E ypT.y/X^T^ - (^ s AjB T - BX s Xj)] ds 
+ I Y, v r X~ s dW s Xj + I X s dWjy^ T ^ T , 

J t J r 

where 

v v V a " x ^ ) 

Let /(«) = E e [X k -i +s \X k -i] and = E e [X k -i +a X^_ 1+s \X k -i], s > 0. Then for any t > 

h(t) = X^iX^ + jf* {/( S )A T + A/( S ) T - (M S )S T - Bfc(*)) + f CT?/ Q l(S) a 2f 2{s) ) ( 3 - 8 ) 
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We can get h(t) by solving (|3.8jl . 

h(t) = e- m X k . x Xl s)A T +Af(sf) 



For 



we can easily get 



Thus 



where 



,-B{t- s ) ( 

^ o\h{s) 

f(t) = e- Bt X k - 1+ I ' e- B WAd8, 
Jo 

/(t)/(t) T = e- Bt X k ^Xl_ x e- BTt + /' e- B (*- 8 )(/( S )A T + ^D^^W 

Jo 

«(* fc _i;0) = - /(1)/(1) T = ofm + a 2 2V2 , 



(3.9) 



/V b < i ->( V BT(1 - s) ^- 
io v m*) / 



Before giving the estimator, we give the following important definition. 

Definition 3.1 Assume A = (ai, . . . , a m ) is n x m matrix, where {a^ : i = 1, . . . , m} are n x 1 vectors, 
and define a operator Vec from a space of matrixes to a space of vectors, i.e. 



( ar \ 



Vec (A) = 



\ a m J 



Now we consider (|3.9f) as a regression equation with {X k — m(Xk-i; A, B))(Xk~m(Xk-i; A, B)) T being 
the response, o~\r\\ -\-o~\r\i being the predictor, and (p\, <r|) being the unknown. Recall {gk '■ k = 1, . . . , n}. 
The WCLSE of (c^of) can be the minimizer of the following sum of squares, here we must of course 
substitute (A, B) by the corresponding estimates 

n 

^g k Vec(Z k - (affji,n + o-lm,n)) T Xcc{Z k - (of^n + a\f\ 2 , «)), 
fe=i 

where {j]\, n ^T]i,ri) is (??i,??2) with (A, i?) substituted by (j4 ra ,i? n ) and 

Z fc = (X fc -m(X fe _ i; i n ,S„))(X fe -m(X fc _ i; i„,5„)) T . 
In particular, those estimators of a\ and o~\ are given by 

-2 yil.n — ¥>12,n 



and 



•l,n 



"2 
°2,n 



V21,n — ^22,; 



(3.10) 



(3.11) 



(i 



where {f)i^ n ,f]2. n ) is (^i , ^72) with (A, B) substituted by (A n ,B n ) 



and 



^12/ 



¥>21,' 



1 

- V" fffcVec(Z fe ) T Vec(77i in )Vec(f} 2 , n ) T Vec(772,n), 
fc=i 

1 " 

- V" fffcVec(Z fc ) T Vec(772,„)Vec(^i.„) T Vec(r/ 2 ,n), 
n *— ' 

k=l 
1 " 

- ^ fffcVec(Z fc ) T Vec(772,„)Vec(77i.„) T Vec(77i in ), 

k=l 
1 ™ 

- V 5feVec(Z fc ) T Vec(77i i „)Vec(^i i „) T Vec(f72,„) 



k=l 



ipn = 9n 



Vec(77i in ) T Vec(?7i,„)Vec(772, n ) T Vec(^2,n) - (Vec(^i in ) T Vec(7?2,n))* 



(3.12) 
(3.13) 
(3.14) 
(3.15) 

(3.16) 



If choose g(x) = 1, we will get the CLSEs respectively. So in the later of this paper, we just need to 
deal with WCLSEs. If the observations of X t arc too large, we can choose g(x) = , where \x\ is the 
norm of x, this WCLSEs are usually discussed in other papers for its good properties. 



4 Consistency and asymptotic normality of [A n , B n ) and (<7^ n , a^n) 

In this section we devote to show that (A n , B n ) and (erf n , erf n ) are strongly consistent. Further, we also 
analyze the central limit theorem of the WCLSEs. All the proofs will be given in Section 5. 

Theorem 4.1 The estimator (A n ,B n ) is strongly consistent, i.e. (A n ,B n ) (A,B) as n — > oo. 

Remark 4.2 The estimators A n , and B n derived solely from the conditional mean function are robust 
against mis specification of the diffusion term in 11. Sty . 

Remark 4.3 In the estimation of (A,B), the diffusion term in hi. Sty can be replaced by o~(X t )dWt, 
where er(-) is an arbitrary function such that the induced stationary distribution of {X t } has finite second 
moment. 



Theorem 4.4 The estimator (<5i n ,02 n) are strongly consistent, i.e. (p\ n i^\ 



[p\,a\) as n — > oo. 



Remark 4.5 From the proof of Theorem 4-4 * n Section 5, we can see any weakly consistent estimator 
of (A, B) gives weakly consistent estimator o/E. 

Let 6 n = (a,i, n , a,2, n , bix,n, 6i2,nj fei.nj &22,nj &2,n) is the WCLSE of 9 given in the last section. 
Now we analyze its asymptotic normality. 

According to the argument above, we know 9 n is the unique solution of the following equation: 



Gn(0) =Y J 9k{w {X k _ 1 ;6){X k -p- 1 X k _ 1 ) + Wl (X k ^;0)Vcc(Z k - (a 2 im + ofo))} = 0, (4.1) 
fe=i 

where 

/ dpi dpi \ 

dai dai 

dpi dpi 

da2 dai 

dpi , ( dm dji2\ dpi , ( dm dj22\ 

dbu v dbn ' dbn dbn ^ V dbu ' dbn 1^ 

dpi I ( din dm \ dpi i ( dm dm \ 

dbu "T" V dbu ' dbu 1^ dbu V dbu ' dbu ^ 

dpi I ( din din s dp 2 , ( dm din \ 

9fc 2 i ^ <9& 2 i ' db 2 i ' ^ <9& 2 i ' <%2i I 

dpi I / &711 ^712 \ dpi 1 / 0)721 ^722 \ 

sfc,., ' v a& 22 ' db 22 ; x 36,, v as,,, j at,, M 




w (a;;^) 



■ O622 ' 9fc 22 ■ 
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and 



w x (x\9) = ( 0, 0, 0, 0, 0, 0, Vecfa), Yec( m ) ) 



The key to the analysis of 9„ is that G n {9) is a Pg-martingale with respect to {.^ n '■ n > 0}. It's 
obvious, so we will not give the proof. 

Let 

m (x; 9) = E 9 [(X k - m(X fc _i; 9))Vec(Z k - v(x; 9)f\X k _ 1 = x] 

and 

ln{x-9) = E e [Vcc(Z fc - v(x;9))Vec(Z k - v(x;9)) T \X k ^ = x}. 



Theorem 4.6 We have y/Ti(9 n - 0) Af(0, V^WV^) as n -too, where 



W = E 6 



gl{w (X ; 9)v(X ; 9)w^(X ; 9) + Wl (X ; 9)^(X ; 9)wJ(X ; 9) 
w (X ; 9)n 3 (X ; 9)wJ(X ; 9) + Wl (X ; 0)fiJ(X o ; 9)w^(X Q ; 9)} 



(4.2) 



V = -E 6 



9l {w a (X ; 9){ dm{ ^ e) ) + Wl (X ; 9) (^Vec(v(X ; 9))) } 



89 ' ' ~^""'"'V90 
By taking g = 1 one can see that all the conclusions above also hold for the CLSEs. 



(4.3) 



5 Proofs 

In this section, we will give the proofs for the theorems in Section 2 and 4 
Proof of Theorem 2.2 Let u(t, A) be the unique solution of the linear equation: 

4ui(£, A) = -b n ui(t, A) + b 12 u 2 (t, A), 



u 2 (t, A) = b 2 iu 1 (t, A) - b 22 u 2 (t, A), 



(5.1) 



with the initial condition u(0, A) = A G D. By applying the comparison theorem to (|2.3[) and (|5.1|) . we 
have Vi(t, A) < Ui(t,\) for i > 0, where i = 1,2. By solving (|5.ip . we get u(t, A) = e~ Bt \. Let £1,62 > 
be the eigenvalues of 5. If £1 = £2, then &n = fo 2 2 > 0, 612 = 621 = and 



u(t, A) 



e- 6llt Ai 
e- h22t A 2 



(5.2) 



If £1 + 6, let ^ = {i : (5 - = 0}, where i = 1,2. Then V 1 ,V 2 are subspaces of K 2 . For any A G D, 
there exit l\ e Li and l 2 G L 2 , such that A = £1 + ^2 and 



By (|5.3|) . we have 



(S- 61)^ = i>l, i = l,2. 
e - Bt A = e-^Hi + e~^H 2 . 



By ([5T21 and (|CT)l , we have 

Vi (t, A) < m(t, A) < C*i(A)e- (?lA « 2) * for some Ci(A) > 0. 



(5.3) 
(5.4) 
(5.5) 



Thus lim vAt, A) = 0. So for V a; £ D, 

t— >-\-oo 



lim / e-^Q t (x,dy)= lim e -(*,««W>-J *<A,i;.(A)>«fa = e - / -<X,«.(A))«fa 



<s 



By (JO), we have (A,v s (X)) = aiUi(s, A) + a 2 v 2 (s, A) < C* 2 (A)e^(« lA « 2 )*, for some C 2 (A) > 0. Then 

/ (A, u s (A))ds < oo for V A G D. 

JO 

Thus, there is a unique distribution Qoo(') satisfying 

j^e-^Q^dy) =exp|-^ (A, w s (A))ds| . 

□ 

Proof of Theorem 4.1 By the relationship of (A n , B n ) and (p n ,^fn), we just need to prove the consis- 
tency of (p n ,%), i-e. 

(Pn,%) ^> (p,l) as n ^ oo. 

By ergodicity, we have 

Ti >n -^-4 Cov((;iXi, 3iXo ) as n — >• oo 

and 

7\,n — > Cov(</i, ^iXiAf^) as n — > oo. 

For 

Cov( 5 iX 1)5l X ) = E e [ 5 2 Ee (X 1 |^'o)^ T ]-E e [ 5l E e (X 1 |^ )]E e [ ffl X T ], 
Cov( 5 i,5i^i^) - E e [ 5 2 Ee (X 1 |^'o)Ar o T ]-E e [« ?1 ]E e [ 3l E e (X 1 |^ )X T ], 

then we get 

Ti,n - Ti,n ^> 7{E [.9i]E e [ 5 i^ o X o T ] - E 9 [ gi X ]E e [giX^]} as n -> oo. (5.6) 
Similarly, we can easily get 

f 2 , n " r 2 , n ^ E e [. gi ]E e [ 5l X X T ] - E e [. gi X ]E e [ ffl X T ] as n ^ oo. (5.7) 
By dHJ), (jSU and ijO) . 



7„ —4 7 as 7i — > oo. 

Similarly, by ergodicity and (|3.5j) . 

P« — — ^ pasn-> oo. 



Proof of Theorem 4.4 By ergodicity and theorem 4.1, we can get 

(Vec(77i, n ), Vec(r72,n)) ^» (Vec(??i), Vec(?7 2 )) as n ^ oo. 

Next we will prove "0 n and y>jj,n converge almost surly, where i, j = 1, 2. Fix 0. For Vi? € [0, oo) 8 , let 

y; t?) = g(x)Vec((y - m(x; - m(a;; 7?)) T ) T Vec(7 7l (79))Vec(77 2 (79)) T Vec(7 ?2 (?9)) 

and U C [0, +oo) 8 be a neighborhood of 9 such that 

E 9 [sup|/i(Xo,X i; tf)|] <oo. 
fee/ 

By ergodicity, (|3T2|) and Theo rem 4.1 
]_ " 

lim ipn,n< lim -y]sup/i(X fc _i,Af fe ;??) = ~E e [sup h{X , X^tf)}. 

n^oo n^oo 77, Jtfgj/ $ eU 



□ 
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Let U | {6>}. Then 

lim" tpu,n< EolKXo^ue)]. (5.8) 

Similarly, we can prove 

lim p 1 i,„>E fl [fc(* o ,.Y 1 ;0)]. (5.9) 

n— J-oo 

By (EH) and (1531) . 

<£>ii,n ^> cTiE fl [5iiVec(?7i) T Vec(?7i)Vec(?72) T Vec(?72)] 

+a 2 2 E e [ ffl Vec(7? 2 ) T Vec(77 1 )Vec(77 2 ) T Vec(7 ?2 )] as n — > oo. (5.10) 
Similarly, when u->cowe can get 

9?i2,„ ^> t^E^iCVecfai^Vecfo)) 2 ] + a 2 2 E [. 9l Vec(7 72 ) T Vec(77 1 )Vec(77 2 ) T Vec(7 ?2 )] (5.11) 

and 

t/) n ^ E e [. 9l Vec(7 7l ) T Vec(7 7l )Vec(7 ?2 ) T Vec(7 ?2 ) - gi(Vec( m ) T Vec( m )) 2 ]. (5.12) 
By m , H53QD , (I5TTTD and dSHD , 



^ 2 a.s. 9 

a! n — ^ (J ^ as n — y oo . 



Similarly, we can prove 



, 2 a.s 2 



'2,n 



a-, as tl — y OO . 



□ 



Proof of Theorem 4.6 Fix 9. By Theorem 4.1 and 4.4, making a Taylor expansion of G n about 9, it 
is enough to show that 

G n (9') = G„(0) + G' n (8)(9' -9) + ^G'^){B' - 0)(0' - 9) T , (5.13) 



where £ is between 6* and 9. Each element of (|5.13[) is 

d 

1 1 = 1 j=l 1 J 



f) 1 £)2 

G nii (0 ' ) = G nii (9) + J2 M-GnM (*,' -fii) + TE TmW G - (0 (°' 3 - °i ) (*/ - ' ) ' 



where i = 1, . . . , 8. 

If choose # = n in the above equation, we will get 

1 d 



-L Gn A0) = £ ^ - + i-£ aW^^ ~ ^ (5 - 14) 

v i=i 1 j=i 1 j 



Let 

?n,2 = Vn{&n.i — @l), Y n _i = -=G n i{9), 

\ n 



Id 1 8 9 2 

3 = 1 3 

Then (j544|) turns to be 



D n T n = Y n , (5.15) 

where D„ = (AmOsxs, T n = (T n ,i, . . . , X; i!8 ) T , Y„ = (Y^,i, . . . , Y ntS ) T . Next we will prove the following 
results respectively, 

D n -^-4 V as n — > oo (5.16) 
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and 



Y n A Af(Q, W)asn^ oo. 
By ([535]) . (pTll)|) and (|CT7)) . we can get 

T„ AA(0, F" 1 W _T ) as n ^ oo. 

(1) Firstly, we prove (|5.16p holds. Let 

h(x, y; 9) = wq(x; 9){y — m(x; 9)) + u>i(x; 9)Vec(z — v(x; 9)), 
where z = (y — m(x; A, B))(y — m(x; A, B)) T . Then (|4.ip turns out to be 

n 

G n (6) = Y J 9kHX k - l ,X k ;6). 

It's easy to prove that 3H(x,y) and 9 € U, such that 

(i) \-^oJ h i(x,y,9)\ < H(x,y) uniformly in U, for = 1,...,8. 

(ii) E fl [|ff(X ,Xi)|] <oo. 
Then 



(5.17) 



1 8 
2^51 d 

4=1 1 J 



if 



< 



j=l k=l 
o 1 n 



3=1 



fe=l 



By Theorem 4.1, 4.4, and 

n 

— Y,\9k\H{X k _ u X k ) ^ lE e [\gi\H(X ,X m )} < 

k=l 

we can get 

d 2 



oo as n — > oo, 
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3 = 1 ] 



as n — > oo. 



By 6U, 



1 ™ f? 

G„.,(0) = - y^g k {(—woAXk-i;9))(X k -m(X k ^;9)) 
n f-^ I o9i 



k =i 



+w ,i{X k - 1 ;6)(-—m(X k _ 1 ;6)) 
+(^-wi,i(X k -i;9))Vec(Z k - v(X k - 1 ;0)) T 

OVi 

+w 1 , i (X k _ 1 ;e)Vec({—^-m(X k . 1 ;6))(X k - m{X k „ i; 9)) T 
+{X k -m{X k „ 1 -9))(—^-m{X k „ 1 -9) f - A w ( Xfc _ i; 0)) }, 



where {wq^, w\ i : i = 1, . . . , 8} are the row vectors of wq and w\. By ergodicity, we calculate each part 
respectively, we can get 

--^G n!i (9)^Vi, h 
n o9i 
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where 

Vi,t = -E e gi[w 0ti (X -,9)—m(X ;6) + w 1 4X 0] 6)Vec(—v{X -,6))] 

l OVi CWi 

Thus wc get (JUTS). 

(2) Secondly, we prove f|5 . 1 T[) holds. Recall {G n (8)} is a martingale, let 

1 1 ™ 

-K = - VE ( [( 5i l l (In,l t ;fl))(ft''ft-i,I^)) T |^i] 

k=l 
1 ™ 

= - J2 9k [MXk-i;6)v{X k _ 1 -6)w^{X k ^;6) + w 1 (X k _ 1 -e)^{X k ^-6)wJ{X k - l] 

k=l 

+Wo(X k _ 1 ;0) t i 3 (X k - 1 ; 0)wJ{X k ^; 9) + w 1 (X k _ 1 ;e) f ij{X k . 1 ;e)w^{X k ^ 1 ;0)] . 
By ergodicity, we can easily get 



□ 



— V n — > W as n — > oo. 
n 

By stationarity, For Ve > 

1 " 
fc=i 

= E e[(9i /l (^o,^i;6 l ) T ^(^o,^i;^)l{<,2 /i ( A - 0!Xi;e) T /i ( Xo ^ Xi;e)> „ e 2 } ] as n -t oo. 
Thus, by the martingale central theorem (see, e.g., Durrett, 2010), (|5.17|) holds. 
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